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Abstract 


Humanity’s  quest  to  find  innovative  ways  to  deal  with  difficult,  monotonous  and 
dangerous  activities  has  been  an  ever  evolving  and  unending  endeavor.  The  current  proliferation 
of  robotic  technology  is  just  the  next  step  in  this  evolutionary  sequence.  Both  civilian  and 
military  agencies  alike  are  vying  for  this  new  round  of  technology.  Most  civilian  applications  of 
robots  are  innocuous  and  generally  perform  menial  tasks.  The  same  cannot  be  said  for  the 
military.  Currently  there  are  numerous  systems  in  each  branch  of  the  military  that  have  some 
autonomous  lethal  engagement  ability.  As  military  professionals,  we  have  a  duty  to  ensure  the 
legal  framework,  proper  policy,  moral  and  ethical  considerations,  as  well  as  proper  tactics  and 
doctrine  are  in  place  to  ensure  compliance  with  the  Rule  of  Engagement  (ROE)  and  the  Laws  of 
Armed  Conflict  (LOAC)  before  embarking  down  a  path  to  fully  automated  autonomous  lethal 
force.  This  paper  will  investigate  some  of  the  more  pressing  issues  and  present 
recommendations  for  potential  paths  forward.  To  facilitate  the  discussion,  the  paper  is  divided 
into  three  major  areas:  the  legal  implications,  ethical  implications  and  professional  implications 


of  use  of  robots  in  warfare. 


Introduction 


The  quest  for  automation  in  the  human  experience  is  as  old  as  time  itself.  Although  the 
technology  has  changed  through  the  ages,  the  concept  that  man  can  take  previously  dangerous  or 
tedious  tasks  and  transform  them  by  use  of  the  information  and  resources  at  his  disposal  is  an 
unending  quest.  Astonishingly,  the  initial  discovery  of  what  would  become  semiconductors 
dates  as  far  back  as  1833,  when  the  now  famous  physicist  Michael  Faraday  stumbled  upon  a 

unique  observation. 1  Faraday  discovered  that  electrical  conduction  increases  with  temperature  in 
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silver  sulfide  crystals,  which  is  the  opposite  of  that  observed  in  copper  and  other  metals. 
However,  it  was  not  until  the  1960s  and  70s  with  the  introduction  of  the  integrated  circuit,  made 
of  the  silicates  first  discovered  by  Faraday,  that  brought  the  revolution  in  computer  technology 
which  has  fundamentally  transformed  the  human  experience.  In  1965,  a  man  by  the  name  of 
Gordon  Moore,  then  Fairchild  Semiconductor's  Director  of  Research  and  Development  and 
future  Intel  co-founder,  postulated  the  number  of  transistors  per  chip  would  double  every  two 
years.4  This  concept  was  subsequently  termed  “Moore’s  Law”  and  has  been  amazingly  accurate 
over  the  past  50  years,  in  large  part  because  chip  manufacturers  build  their  future  development 
roadmaps  based  on  this  theory.5  Now  in  2015,  computer  technology  has  become  so  advanced 
that  recently  a  savvy  16  year-old  put  the  Windows  95  operating  system  on  an  Android 
smartwatch! 6 

The  giant  leaps  in  technology  during  the  past  decade  have  also  led  to  some  very 
sophisticated  robot  technology.  In  fact,  the  2013  Defense  Advanced  Research  Projects  Agency 
(DARPA)  Robotics  Challenge  showcased  some  of  the  most  advanced  robots  seen  to  date.7  The 
winning  robot,  SCHAFT,  developed  by  Japanese  company  SHAFT  Inc.,  attained  an  impressive 
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27  of  32  possible  points  during  the  challenge.  The  challenge  consisted  of  eight  distinct  tasks: 


driving  a  vehicle,  maneuvering  the  robot  over  rough  terrain,  going  up  an  industrial  ladder, 
removing  debris  from  a  pathway,  opening  different  types  of  doors,  cutting  through  a  wall  in  a 
certain  pattern,  closing  different  types  of  valves,  and  finally,  unwinding  and  connecting  a  hose  to 
a  spigot.9  While  the  DARPA  robots  are  intended  for  peaceful  and  innocuous  purposes,  there  are 
other  robots  being  developed  for  uses  that  are  more  threatening. 

There  are  many  governments  and  associated  defense  contractors  working  on  various 
robots  for  use  in  combat  situations.  Some  of  these  machines  have  the  ability  to  autonomously 
target  and  engage  adversaries.  Recently,  Rear  Admiral  Matthew  L.  Klunder,  Chief  of  United 
States  Naval  Research,  debuted  a  new  swarming  boat  technology  where  as  many  as  20-30 
autonomous  boats  can  be  designated  to  surround  a  single  target. 10  The  boats  can  carry  a  number 
of  different  payload  options  including  spotlights,  high  powers  speakers,  or  even  offensive 
weapons. 1 1  Of  course,  the  U.S.  Navy  is  not  the  only  service  researching  autonomous  weapon 
systems.  The  U.S.  Air  Force  and  Army  also  have  active  programs.  The  Air  Force  developed  the 
MQ-9  Reaper  and  deployed  it  successfully  to  both  Iraq  and  Afghanistan. 12  Additionally,  the 
U.S.  Anny  has  numerous  systems  in  development  including  the  unmanned  robocopter  sniper 
system,  Autonomous  Rotorcraft  Sniper  System  (A RSS ), 1  ’  and  other  ground-based  systems  that 
do  everything  from  deliver  foodstuffs14  to  QinetiQ’s  Modular  Advanced  Armed  Robotic  System 
(MAARS)  which  can  carry  multiple  combat  loads  including  a  40mm  grenade  launcher  and 
7.62mm  machine  gun. 15  The  emergence  of  these  systems,  especially  those  capable  of 
autonomous  lethal  engagement,  bring  with  them  many  questions:  how  should  we  test  and  verify 
such  systems,  how  should  they  be  employed,  who  is  responsible  if  the  autonomous  systems 
malfunction  and  harms  an  innocent  human,  and  ultimately  should  we  even  develop  and  use 


autonomous  lethal  force? 


Although  the  technology  to  produce  robots  similar  to  Terminator  or  I,  Robot  are  still 
decades  away,  it  is  undeniable  that  future  autonomous  lethal  systems  will  be  used  in  armed 
conflict.  In  fact,  the  first  iteration  of  such  systems  are  already  in  use  in  combat  today  as 
discussed  previously.  As  military  professionals,  we  have  a  duty  to  address  carefully  the 
questions  posed  above  and  ensure  the  legal  framework,  proper  policy,  moral  and  ethical 
considerations,  as  well  as  proper  tactics  and  doctrine  are  in  place  to  ensure  compliance  with  the 
Rule  of  Engagement  (ROE)  and  previously  mentioned  concepts  before  embarking  down  a  path 
to  fully  automated  autonomous  lethal  force.  This  paper  will  investigate  some  of  the  more 
pressing  issues  and  present  recommendations  for  potential  paths  forward.  To  facilitate  the 
discussion,  the  paper  is  divided  into  two  major  areas:  the  legal  and  ethical  implications  of  use  of 
robots  in  warfare. 

Definitions 

Before  delving  into  the  details  of  the  discussion,  a  basic  sense  of  the  technologies  and 
ideas  we  are  discussing  are  in  order.  There  are  many  different  types  of  robots  or  automated 
machines.  Consequently,  although  some  specific  technologies  will  be  described,  they  are 
primarily  for  illustrative  purposes  and  should  not  be  construed  to  represent  the  totality  of  the 
robotic  family  under  question.  Additionally,  for  the  purposes  of  this  paper,  I  will  use  the 
definition  of  a  robot  set  forth  by  Lin,  Bekey  and  Abney  in  their  paper  for  the  Office  of  Naval 
Research  titled,  Autonomous  Military  Robotics:  Risk,  Ethics,  and  Design : 

Robot:  A  powered  machine  that  (1)  senses,  (2)  thinks  (in  a  deliberative,  non-mechanical 
sense),  and  (3)  acts.16 

This  definition  will  exclude  lethal  weapons  such  as  land  mines,  missiles  and  grenades, 
although  all  of  them  have  some  autonomy  or  ability  to  interact  with  the  environment.  The  key 


distinction  being  the  second  part  of  the  definition  dealing  with  the  required  deliberative  and 
logical  decision  process.  In  other  words,  we  are  primarily  concerned  with  machines  that  can 
autonomously  operate  and  interact  with  the  world  around  them  and  make  decisions  to  use  lethal 
force. 

The  robots  described  in  this  paper  may  or  may  not  be  fully  mobile,  but  can  maneuver  in 
some  fashion.  In  addition,  many  autonomous  machines  involve  a  spectrum  of  man-in-the-loop 
(MIL)  interaction  and  this  paper  is  not  meant  to  exclude  such  technology.  In  other  words,  most 
robots  require  at  least  some  human  interaction,  even  if  it  is  to  give  the  initial  orders. 

Legal  Implications 

Most  discussions  on  the  use  of  robots  in  war  begin  with  legal  implications  of  their  use 
during  combat.  The  preferred  and  most  appropriate  framework  utilized  is  Just  War  Theory  or 
more  broadly  understood  as  the  Laws  of  Armed  Conflict  (LOAC). 17  These  laws  have  evolved 
over  time  and  are  concerned  with  the  conduct  before,  during  and  after  war;  known  more 
commonly  as  Jus  ad  Bellum,  Jus  in  Bello  and  Jus  post  Bellum. 

Jus  ad  Bellum 

1  o 

Jus  ad  Bellum  are  laws  to  prevent  the  start  of  war.  They  describe  the  conditions  under 
which  war  can  be  legally  and  morally  justified,  a  legitimate  war. 19  Although  there  is  no 
universally  accepted  list  of  conditions  for  Jus  ad  Bellum,  they  are  generally  split  into  seven 
areas:  a  just  cause,  proportionality,  a  legitimate  authority  to  declare  war,  a  reasonable  chance  of 
success,  right  intention,  war  declared  as  a  last  resort  and  the  goal  of  a  just  peace.-  Some  of  the 
current  literature  dealing  with  autonomous  lethal  force  assumes  the  Jus  ad  Bellum  conditions 
will  be  met  and  thus  do  not  focus  much  on  this  section  of  LOAC.  However,  there  are  potential 
issues  when  dealing  with  autonomous  lethal  force. 


In  his  Air  War  College  paper  on  this  same  subject,  Colonel  Michael  Contratto  aptly 
pointed  out  the  potential  for  autonomous  systems  to  circumvent  the  LOAC  rules  given  our 
reliance  on  systems-of-systems  and  thus  illegitimately  begin  a  war.  He  further  states  that  even 
if  the  system  followed  the  Rules  of  Engagement  (ROE)  and  LOAC,  there  is  potential  for  an 
autonomous  system  to  act  without  the  authority  of  the  legitimate  owner,  once  again  leading  to 
potentially  disastrous  consequences.  This  argument  is  especially  compelling  given  the 
incredibly  complex  nature  of  the  autonomous  systems  we  are  discussing,  the  insurmountable  task 
of  completely  testing  all  possible  input  and  outputs  of  the  system  and  finally  the  dire 
consequences  of  system  failure,  notably  the  potential  loss  of  life  and  destruction  of  property. 

Contratto  is  not  alone  in  his  misgivings  of  autonomous  lethal  force  and  the  Jus  ad  Bellum 
argument.  Peter  M.  Asaro,  a  well-known  philosopher  and  computer  scientist  in  the  robotics 
community,  has  written  extensively  on  the  use  of  autonomous  lethal  force  in  combat.  One  of  his 
arguments  against  the  use  of  robots  includes  lowering  the  barriers  to  entry  into  war.24  By  using 
them  there  could  be  a  dramatic  reduction  in  the  cost  of  human  capital  on  the  side  deploying  the 
robots,  a  significant  barrier  to  entry  into  war  and  potentially  cutting  at  the  proportionality,  right 
intention  and  war  declared  as  a  last  resort  aspects  of  Jus  ad  Bellum.  Others,  namely  Patrick 
Lin,  George  Bekey  and  Keith  Abney,  do  not  find  this  argument  compelling,  citing  the  fact  that 
robots  themselves  do  not  necessarily  change  the  calculus  of  the  individual  Jus  ad  Bellum 
requirements.26  Additionally,  they  fittingly  point  out  this  argument  could  be  used  to  prevent  the 
advancement  of  any  military  technology,  a  point  not  completely  lost  on  Asaro.  ’  Finally,  they 
argue  that  a  country  could  develop  a  deterrence  strategy  with  an  anny  of  robots,  similar  to 
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nuclear  weapons,  where  the  mere  possession  of  such  weapons  significantly  deters  attack. 
Regardless  of  one’s  particular  point  of  view  on  the  preceding  arguments,  the  mere  fact  that  they 


are  debated  should  be  evidence  enough  of  the  need  to  thoroughly  investigate  the  Jus  ad  Bellum 
requirements  relative  to  autonomous  lethal  force.  It  is  this  author’s  opinion  the  use  of  a  robot 
army  for  autonomous  lethal  force  is  a  revolution  in  military  affairs  and  would  transfonn  the  basic 
fabric  of  warfare. 

Interestingly,  with  this  transformation  on  the  horizon,  one  would  expect  to  find  a  wealth 
of  professional  articles  or  other  literature  from  the  military  and  governmental  organizations. 
Unfortunately,  this  is  not  the  case.  Although  there  are  certainly  papers  from  the  professional 
military  education  institutions,  the  dearth  of  them  is  surprising.  Of  more  concern  is  the  lack  of 
ethical  considerations  in  the  few  official  government  documents  currently  available  dealing 
specifically  with  autonomous  machines.  In  his  book,  Wired  for  War,  Singer  laments  the  lack  of 
serious  contemplation  of  these  ethical  considerations  at  a  2006  conference,  “Rethinking  the  US 
Military  Revolution,”  and  a  2007  conference  including  100  international  law  experts  discussing 
“New  Battlefields,  Old  Laws.”30  Additionally,  in  the  2007  Unmanned  Systems  Safety  Guide  for 
DoD  Acquisition  (most  current  version),  there  is  not  a  single  reference  to  ethics  in  this 
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document,  where  one  would  certainly  expect  to  find  at  least  a  cursory  discussion.  Finally,  in  a 

recent  update  to  the  Air  Force’s  remotely  piloted  aircraft  roadmap,  USAF  RP A  Vector:  Vision 

and  Enabling  Concepts  2013-2038,  there  are  only  two  references  to  ethics  in  the  100-page 

document.  The  first  states  the  following: 

With  appropriate  CONOPS  and  doctrinal  considerations,  the  future  potential  for 
autonomous  systems  to  independently  select  and  attack  targets  with  lethal  effects  exists 
from  a  technology  perspective.  To  achieve  this,  the  Air  Force  and  DoD  must  first 
address  the  legal,  moral,  and  ethical  concerns  that  autonomous  lethal  effects  present 
as  well  as  consider  minimum  safeguards.  Future  RPA  may  continue  on  mission  using  a 
combination  of  autonomous  behaviors  and  assured  position,  navigation,  and  timing 
(PNT)  if  communications  are  lost  or  degraded  in  an  A2/AD  scenario,  (emphasis  added) 


It  is  encouraging  to  see  the  Air  Force  acknowledge  the  need  to  address  the  “legal,  moral,  and 
ethical  concerns”  with  autonomous  lethal  force,  but  it  is  unclear  what  exactly  is  being  pursued. 
The  second  reference,  innocuously  inserted  into  the  nuclear  strike  section  states,  “Ethical 
discussions  and  policy  decisions  must  take  place  in  the  near  term  to  guide  the  development  of 
future  UAS  capabilities,  rather  than  allowing  the  development  to  take  its  own  path  apart  from 
this  critical  guidance.”34  It  is  not  explicitly  clear  whether  this  statement  is  directed  specifically  at 
nuclear  RPA  operations  or  more  broadly  dealing  with  general  RPA  operations,  and  one  would 
certainly  hope  that  weapons  as  destructive  as  a  nuclear  bomb  might  never  be  armed  on  an 

TC 

autonomous  vehicle.  Either  way,  it  is  ironic  given  the  current  state  of  RPA  development  and 
the  lack  of  credible  and  substantive  guidance  on  the  ethical  use  of  autonomous  lethal  force 
throughout  the  Department  of  Defense.  These  references  provide  an  indication  that  the  United 
States  military  and  policy  makers  have  much  more  work  to  do  for  a  true  Jus  ad  Bellum  case  for 
the  use  of  autonomous  lethal  force. 

Jus  in  Bello 

tz: 

Jus  in  Bello  refers  to  the  proper  conduct  in  or  during  war.  It  applies  legal  and  moral 
restraints  to  the  conduct  of  war,  and  although  no  complete  set  of  agreed  upon  principles  exist, 
they  are  generally  divided  into  two  main  areas:  discrimination  and  proportionality.  Most  of  the 
literature  surrounding  autonomous  lethal  force  deals  with  issues  surrounding  these  principles. 
Discrimination  obligates  the  belligerents  of  a  conflict  to  discriminate  between  military 

TO 

combatants  and  the  civilian  population.  Force  can  then  only  be  applied  to  the  military 

TO 

combatants.  In  theory,  one  could  argue  that  advancements  in  robotic  technology  could 
someday  allow  an  autonomous  machine  to  distinguish  properly  between  legitimate  and 
illegitimate  targets.40, 41  However,  in  the  fog  and  friction  of  a  wartime  environment,  this  will  be 


a  daunting  task,  and  many  believe  it  is  almost  insurmountable  due  to  the  very  nature  of  war, 
especially  in  the  case  of  insurgencies  where  the  enemy  combatant  is  virtually  indistinguishable 
from  the  surrounding  populace.42  Laying  aside  the  technical  difficulties  and  assuming  that  an 
autonomous  robot  could  make  this  distinction  based  upon  proper  coding  of  the  LOAC  and  the 
ROEs,  it  is  then  reasonable  to  assume  an  autonomous  robot  could  adequately  discriminate 
between  targets. 

Perhaps  the  more  important  question  is  whether  an  autonomous  robot  could  discriminate 
better  than  a  human  could.  In  his  Naval  War  College  paper,  Michael  A.  Guetlein  aptly  points  out 
that  discrimination  would  ultimately  come  down  to  an  adjustable  probability  calculation  based 
on  the  ROEs  and  presence  of  civilians.43  On  an  autonomic  level,  this  is  how  humans  engage 
their  own  decision  process;  that  is  one  of  probabilities.  Additionally,  Dr.  Ronald  C.  Arkin,  a 
roboticist,  roboethicist  and  Regents’  Professor  at  Georgia  Tech,  argues  that  eventually  robots 
will  be  able  to  “perform  more  ethically  than  human  soldiers  are  capable  of.”44  However,  given 
the  sheer  complexity  of  the  task  and  limits  of  current  technology  as  well  as  systems  integration, 
this  capability  is  still  decades  away,  if  it  ever  becomes  possible  to  have  a  machine  truly  able  to 
autonomously  make  such  moral  decisions. 

Proportionality  seeks  to  avoid  killing  civilians  or  damaging  their  property  unless  military 
necessity  dictates  otherwise  and  only  if  the  proportional  military  gain  exceeds  the  cost  of  the 
civilian  casualties.45  At  first  glance,  it  appears  the  proportionality  principle  might  be  easier  for 
autonomous  robot  compliance.  In  fact,  Guetlien  offers  only  one  passing  sentence  on  the  issue, 
“Due  to  AW  [autonomous  weapon]  logic  constraints,  it  can  be  easily  argued  that  autonomous 
weapons  will  be  more  proportional.”46  (emphasis  added)  Additionally,  in  the  study  sponsored  by 
the  Office  of  Naval  Research  they  make  the  following  statement,  “After  testing,  it  is  easy  to 


imagine  that  robots  could  perform  at  least  as  well  as  humans  in  deploying  no  greater  violent 
force  than  needed,  and  thereby  passing  the  ‘military  Turing  test’  for  moral  deployment.”47 
(emphasis  added)  In  Arkin’s  100-page  foundational  essay  on  robotic  ethics,  he  goes  much 
further  in-depth  on  the  issue  and  spends  numerous  pages  describing  how  a  robot  could  be 
programmed  via  set  theory  to  ensure  both  discrimination  and  proportionality.  However,  it  is 
not  clear  that  the  proper  discriminating  sensor  inputs  are  currently,  or  would  ever  be  available  to 
feed  such  algorithms.  This  argument  is  precisely  what  Sharkey  sets  forth  and  further 
underscores  the  idea  that  there  are  no  objective  measures  to  detennine  proportionality.49 

Although  not  specifically  delineated  in  either  discrimination  or  proportionality,  many 
have  fittingly  argued  that  if  an  ethical  autonomous  robot  were  possible,  it  has  some  unique 
characteristics  that  make  it  especially  suited  for  the  battlefield.  For  instance,  robots  do  not  get 
tired,  angry,  have  feelings  of  revenge,  succumb  to  boredom,  or  a  host  of  other  undesirable  human 
traits,  which  can  lead  to  egregious  violations  of  the  ROEs  and/or  LOAC.  Admittedly,  the 
instances  of  such  occurrences  are  low  in  present  conflicts  especially  given  the  number  of  fielded 
soldiers  and  the  length  of  the  conflicts.  However,  when  they  do  occur  they  receive  very 
widespread  attention  across  the  globe  and  cause  deleterious  effects  to  the  prestige  and  political 
standing  of  the  United  States,  as  well  as  feed  the  enemy’s  cause,  which  aims  to  undermine  the 
United  States  intentions  and  legitimacy  in  the  war  effort.  The  Abu  Ghraib  prison  scandal  in  Iraq 
and  the  Maywand  District  murders  in  Afghanistan  are  examples  of  such  unfortunate  events. 

Similarly,  by  deploying  autonomous  robots  we  could  effectively  reduce  the  number  of 
required  human  troops  thus  saving  many  from  the  horrors  of  war.  Some  argue  we  actually  have 
a  moral  obligation  to  do  such  and  even  point  to  United  States  Code,  Title  10,  Sections  3583, 

5947  and  8583,  which  charges  commanding  officers  from  the  four  major  branches  of  military 


service  “to  promote  and  safeguard  the  morale,  the  physical  well  being,  and  the  general  welfare  of 
the  officers  and  the  enlisted  persons  under  their  command  or  charge.”50  Of  course,  the  military 
is  specifically  organized,  trained  and  equipped  to  fight  the  nation’s  wars  and  thus  expected  to  be 
placed  in  hann’s  way.  It  seems  injudicious  to  use  the  Title  10  code  as  justification  for 
autonomous  lethal  force. 

Jus  post  Bellum 

Jus  post  Bellum  refers  to  the  proper  conduct  after  war.51  It  is  concerned  with  items  such 
as  ceasefire  terms,  surrender  terms  and  the  disposition  of  prisoners  of  war  after  conflict  has 
officially  ceased.  “Asa  recent  Office  of  Naval  Research  report  points  out,  many  of  the  same 
issues  in  jus  post  bellum  are  covered  in  jus  in  bello.  However,  there  are  a  couple  unique 
circumstances  which  could  arise  in  light  of  the  use  of  autonomous  lethal  robots.  The  first 
concerns  the  ceasefire  terms.  In  this  case,  consider  two  belligerents  who  go  to  war  each  solely 
with  its  own  autonomous  robot  army.  It  is  reasonable  to  assume  each  side  would  fight  to  the  last 
robot,  something  not  normally  done  with  a  conventional  human  anny  (i.e.  no  rational  nation 
fights  to  their  last  soldier).54  Next,  at  the  conclusion  of  the  hostilities,  would  the  losing  side  feel 
compelled  to  abandon  hope  for  victory  or  further  launch  human  soldiers  into  the  war  effort 
against  the  remaining  robots?55  Additionally,  Asaro  makes  the  point  that  a  war  with  solely  a 
robotic  army  on  each  side  could  appear  as  little  more  than  a  violent  sporting  event  and  some 
political  concession  such  as  territory  would  need  to  occur  to  consider  the  hostilities  profitable.56 

The  second  area  of  concern  is  post-war  stabilization,  which  may  include  a 
counterinsurgency.  As  recent  events  in  Iraq  and  Afghanistan  depict,  winning  the  war  may  be  the 
easy  part,  but  winning  the  peace  is  a  considerably  different  matter.  Some  argue  that  the  use  of 
robots  in  a  post-war  environment  may  be  very  beneficial,  especially  in  preventing  such  things  as 


raping,  pillaging,  taunting,  etc.57  Other’s  point  out  that  ensuring  peace,  especially  in  an 
insurgency,  requires  “winning  the  hearts  and  minds  of  the  people,”  which  a  robot  army  is 
certainly  not  well  equipped  to  perform.  It  appears  cultural  sensitivities  would  have  the  greatest 
impact  on  the  use  of  autonomous  robots  during  a  post-war  occupation. 

Ethical  Implications 

Assuming  the  barriers  to  the  legal  implications  could  satisfactorily  be  overcome,  there  are 
still  ethical  implications  for  the  use  of  autonomous  lethal  force.  I  will  investigate  two  areas  in 
this  section:  moral  agency  and  chivalry. 

Moral  Agency 

The  first  ethical  issue  arriving  from  the  use  of  autonomous  lethal  force  comes  in  the  fonn 
of  moral  agency.  As  human  beings,  we  are  held  accountable  for  our  own  actions  barring  some 
legal  claim  to  temporary  insanity  or  the  like.  During  warfare  this  concept  is  generally  assumed 
away.  In  other  words,  it  is  assumed  that  the  application  of  lethal  force  carries  with  it  an  assumed 
morally  accountable  agent  or  someone  who  can  accept  responsibility  if  civilian  deaths  should 
occur.  59  The  advent  of  fully  autonomous  robots  would  challenge  this  assumption. 

In  modem  warfare,  there  exists  technology  which  significantly  removes  the  human  from 
the  physical  source  of  the  lethal  force  (e.g.  weaponized  RPAs).  However,  there  is  still  a  MIL 
and  moral  agency  is  attributable  directly  to  the  human  controlling  the  machine.  With  a  truly 
autonomous  robot,  the  question  arises,  who  is  ultimately  responsible  for  the  actions  of  such  a 
machine?  Is  it  the  individual  programmer,  the  company  who  developed  the  robot,  the  military 
commander  under  whose  charge  the  robot  falls,  the  military  service  that  requested  the 
development  of  the  technology  or  even  the  United  States  government  at  large  for  authorizing  the 
use  of  such  autonomous  lethal  force  in  the  first  place?  There  does  not  seem  to  be  an  easy  answer 


to  this  question.  Dr.  Robert  Sparrow,  a  Professor  in  the  School  of  Philosophy  and  Bioethics  at 
Monash  University  in  Australia,  is  quick  to  point  out  that  holding  the  programmer  responsible 
for  a  truly  autonomous  robot  is  equivalent  to  holding  parents  responsible  for  their  grown 
children’s  behavior,  which  obviously  militates  against  conventional  legal  thought.60  However, 
generally  speaking,  in  military  organizations  the  military  commander  is  held  accountable  for  the 
actions  of  his  subordinates  even  if  he  or  she  may  not  have  had  the  ability  to  directly  influence  the 
subordinate’s  behavior.  Although  some  find  this  answer  satisfying,  the  next  obvious  question  is, 
should  a  military  commander  be  held  responsible  for  the  acts  of  an  autonomous  robot  for  which 
he  had  no  ability  to  control  the  coding  or  command  influence  on  its  behavior?  As  a  former 
military  commander,  I  find  this  logic  troubling  especially  considering  the  fact  I  would  have 
almost  no  ability  to  influence  an  autonomous  robot’s  actions  and  yet  be  held  accountable  for 
them.  At  least  in  the  case  of  a  human  being,  you  can  legitimately  argue  a  commander  had  the 
ability  to  directly  influence  the  personnel  assigned  to  him  or  her,  especially  at  the  squadron  level. 

Supposing  one  could  argue  a  robot  did  achieve  moral  autonomy  in  a  Kantian  sense  and 
would  thus  be  held  responsible  for  its  own  actions,  how  would  punishment  be  administered  to 
such  a  machine?  The  machine  does  not  have  feelings  like  a  human.  No  amount  of  time  in  prison 
would  alter  the  robot’s  behavior.  Would  recoding  be  an  option?  If  recoding  were  an  option, 
what  about  the  rest  of  the  autonomous  robots?  Would  they  be  recoded  as  well?  Last,  would 
recoding  be  a  de  facto  admission  the  robot  was  not  truly  a  Kantian  moral  agent  to  begin  with? 
Additionally,  the  current  American  legal  framework  serves  as  the  means  through  which  society 
conducts  itself  and  provides  a  certain  deterrent  to  would-be  criminals  in  the  form  of  punitive 
punishment.  Would  other  autonomous  robots  be  dissuaded  from  committing  a  crime  by 


witnessing  one  of  their  own  punished?  All  of  these  questions  raise  considerable  misgivings 
about  assigning  moral  agency  to  an  autonomous  robot. 

Chivalry 

Another  concept  that  raises  ethical  concerns  is  the  idea  of  chivalry.  Chivalry  includes 
waging  war  in  accordance  with  well-recognized  fonnalities  and  courtesies  and  is  occasionally 
listed  as  one  of  the  Jus  in  Bello  principles.61  I  believe  the  concept  of  chivalry  transcends  as  only 
a  Jus  in  Bello  concept.  It  drives  at  the  very  heart  and  nature  of  warfare.  Asaro  gives  one  of  the 
most  compelling  arguments  when  he  states: 

This  approach  to  war  could  be  deemed  unjust  by  traditional  conventions  of  war  because 
those  doing  the  killing  are  not  themselves  willing  to  die.  This  principle  is  fundamental 
because  it  powerfully  influences  our  sense  of  fairness  in  battle,  and  concerns  the  nature  of 
war  as  a  social  convention  for  the  settling  of  disputes.62 

Ironically,  the  advent  of  most  military  technologies,  including  autonomous  robots,  is  to  further 

remove  humans  from  the  horrors  of  war.  This  is  especially  true  in  our  current  age  of  almost 

instant  media  and  the  low  tolerance  for  casualties  (civilian  or  military)  in  democratic  societies. 

However,  to  our  enemies,  the  use  of  such  technology  shows  a  sign  of  weakness  and  may  even 

further  exacerbate  asymmetrical  attacks  such  as  terrorism.  When  viewed  from  a  purely  realist  or 

political  perspective,  one  could  argue  an  autonomous  robot  can  achieve  the  same  goals  as  a 

human  soldier,  thus  there  would  be  no  qualms  with  using  such  technology  and,  in  fact,  would  be 

desirable  given  the  reduction  in  the  potential  loss  of  human  capital.  On  the  other  hand,  from  the 

perspective  of  the  international  community  or  opposing  belligerent,  it  could  be  viewed  as 

cowardly  and  raise  questions  as  to  the  legitimacy  of  the  war  effort  as  well  as  the  state’s 

intentions.  Next,  as  Guetlein  points  out,  “[mjachines  do  not  have  the  capacity  for  the  military 

ethos  of  chivalry  (courage,  justice,  mercy,  generosity,  faith,  nobility,  and  hope).” 


Contratto’s  thesis  rests  on  this  basic  idea  of  chivalry.64  He  ultimately  contends  the 
military  ethos  requires  professional  soldiers  to  do  the  bidding  of  their  nation’s  wars.65  He 
persuasively  argues  autonomous  lethal  force  may  ultimately  degrade  the  status  of  the  military 
from  a  profession  to  that  of  basic  contract  management.66  He  concludes  with,  “[tjherefore  the 
offensive  use  of  ALE  should  always  keep  a  human  in  the  loop  of  the  kill  chain.”  One  could 
counter  argue,  what  is  the  real  difference  if  someone  is  playing  a  "video  game"  war  with  the 
robots  or  the  robots  are  doing  it  themselves?  How  is  the  soldier  any  more  fulfilling  honor  or  the 
warrior  ethos  by  commanding  the  robot  via  datalink?  The  truth  is,  we  are  already  at  a  point 
where  some  soldiers  are  far  removed  from  the  battlefield  (e.g.  Air  Force  RPAs),  and  whose  life 
is  more  in  danger  on  the  car  ride  to  work  than  at  any  time  during  military  operations.  Aside  from 
the  overwhelming  technological  challenges  to  creating  a  truly  Kantian  morally  autonomous 
robot,  chivalry  represents  one  of  the  most  vexing  issues  to  the  use  of  autonomous  lethal  force. 

Recommendations 

Tactical  /  Objective 

1 .  Encourage  professional  studies  papers  at  the  various  military  professional  education 
institutions  on  the  subject  of  autonomous  lethal  force. 

2.  Encourage  the  military  legal  community  to  engage  in  an  in-depth  historical  review  of 
LOAC  and  ROEs,  and  then  determine  recommendations  for  potential  laws  governing 
the  use  of  autonomous  lethal  force. 

3.  Encourage  military  headquarters  staffs  to  require  ethical/moral  sections  in  documents 
governing  procurement  and  vector  documents  dealing  with  systems  that  could  be 


used  for  autonomous  lethal  force. 


4.  Encourage  military  participation  in  robotic  ethics  conferences  such  as  We  Robot, 
International  Conference  on  Artificial  Intelligence  and  Law,  Computer  Ethics 
Philosophical  Enquiry,  etc. 

Even  with  all  of  the  tactical/objective  recommendations  above,  more  must  be  done  at  the 
strategic  level  to  effect  lasting  change,  especially  since  the  concepts  surrounding  the  use  of 
autonomous  lethal  force  deal  primarily  with  states  in  the  international  arena. 

Strategic 

1 .  A  concerted  effort  on  the  part  of  the  United  States  government  to  increase  dialogue 
on  the  subject  of  autonomous  lethal  weapons  needs  to  occur. 

2.  Recognized,  legitimate  international  governing  bodies  such  as  the  United  Nations, 
North  Atlantic  Treaty  Organization,  International  Monetary  Fund,  World  Bank,  etc. 
should  be  engaged  to  come  up  with  international  norms  for  the  use  of  autonomous 
lethal  force  and  potential  ramifications  of  their  misuse. 

Ultimately,  enforceable  decisions  on  the  legitimate  use  of  autonomous  lethal  force  will  be 
made  at  the  strategic  level,  having  been  informed  by  the  items  listed  in  the  tactical/objective 
section. 

Conclusion 

The  age  of  autonomous  lethal  weapons  is  presently  upon  us.  It  is  not  inconceivable  to 
imagine  fully  autonomous  robots  similar  to  those  currently  depicted  in  science  fiction  movies  by 
the  end  of  the  2 1st  Century.  Although  they  still  may  not  be  considered  free  moral  agents,  the 
social  implications  of  such  machines  raises  a  host  of  important  legal,  ethical  and  professional 
questions.  On  the  civil  side,  police  departments  will  face  similar  questions  in  the  coming  years 
as  cities  look  to  use  these  technologies  in  law  enforcement  capacities.  This  paper  has  addressed 


some  of  those  questions  dealing  specifically  with  the  military  application  of  autonomous  lethal 
force. 

We  investigated  some  legal  implications  of  autonomous  lethal  force  in  the  framework  of 
the  Law  of  Armed  Conflict  principles  as  provided  in  Just  War  Theory:  Jus  ad  Bellum,  Jus  in 
Bello  and  Jus  post  Bellum.  Some  of  the  more  pressing  issues  raised  were  the  potential  for 
autonomous  robots  to  lower  the  barriers  for  entry  into  war,  robots  accidentally  starting  wars 
(sovereignty),  the  ability  for  autonomous  lethal  machines  to  properly  discriminate  between 
combatants  and  non-combatants,  objective  measures  for  a  proportionality  algorithm,  and  the 
potential  difficulties  in  terminating  a  robot  vs.  robot  army  war.  Currently,  many  of  these 
concerns  are  assumed  away  inferring  that  eventually  the  technology  will  exist  for  autonomous 
robots  to  perfonn  such  actions  at  a  level  equal  to  or  greater  than  a  human  soldier. 

We  also  discussed  some  overriding  ethical  implications  of  the  use  of  autonomous  lethal 
force.  These  included  moral  agency  and  chivalry.  Although  the  previous  legal  areas  of  concern 
may  someday  be  overcome  by  technical  advancement,  it  is  not  clear  there  will  ever  be  a  day 
where  the  moral  agency  and  chivalry  issues  could  adequately  be  addressed.  Perhaps  the 
international  norms  will  change  allowing  for  the  use  of  such  technologies,  similar  to  how  the 
norms  of  high  seas  survivor  rescue  changed  with  the  advent  of  submarine  warfare.  Ultimately, 
the  question  under  the  ethical  implications  section  is,  should  we  utilize  these  technologies  just 
because  they  are  available?  Are  we  going  to  create  something  akin  to  the  nuclear  weapon,  where 
it  is  considered  mass  suicide  to  employ  such  a  weapon,  yet  countries  have  stockpiles  of  these 
weapons,  which  we  then  have  to  regulate  closely  and  protect  them  from  falling  into  the  wrong 


hands? 


Finally,  I  provided  recommendations  on  how  to  move  forward  responsibly  with  the 
development  and  deployment  of  such  weapons.  Many  industrialized  states  are  well  down  the 
road  to  autonomous  lethal  force  and  some  (United  States,  South  Korea)  have  already  employed 
such  technology.  As  military  professionals,  we  have  a  responsibility  to  inform  policy  makers  of 
the  implications  of  the  use  of  such  force  and  look  for  ways  to  legitimize  the  use  of  autonomous 
lethal  force  on  an  international  level  with  the  proper  legal,  ethical  and  policy  measures  in  place. 
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